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Risk  Management  (Is  Not)  for  Dummies' 


Lt.  Col.  Steven  R.  Glazewski 
Air  Force  Institute  of  Technology 

Software  program  managers  crave  a  silver  bullet  in  the  form  of  a  comprehensive  checklist  of  things  to  watch  so  the  program 
does  not  suffer  from  bad  surprises.  High  lighted  in  this  article  are  some  prime  examples  from  almost  1 5  years’  experience 
acquiring  software  in  Department  of  Defense  programs,  from  identifying  broad  areas  where  software  risks  tend  to  hide  to 
describing  an  eight-step  risk  management  process.  While  there  are  no  silver  bullets  to  be  found,  there  are  a  few  golden 
nuggets  if  you  tnake  the  focused  effort  to  look! 


What  is  risk  management?  We  have  all 
heard  the  saying,  “Give  a  man  a  fish, 
and  you  feed  him  for  a  day.  Teach  a  man 
to  fish,  and  you  feed  him  for  a  lifetime.” 
Let  me  revise  that  from  a  risk  manage¬ 
ment  standpoint:  “Put  out  a  manager’s 
fires,  and  you  help  him  for  a  day.  Teach  a 
manager  fire  prevention,  and  you  help  him 
for  a  career.”  If  a  manager  understands 
good  risk  management,  he  can  worry 
about  things  other  than  firefighting. 

Unfortunately,  most  people  who  look 
for  risk  management  help  are  seeking  to 
know  the  steps  to  put  fires  out.  After  all, 
being  a  good  firefighter  has  its  rewards! 
Take  a  look  at  your  organization’s  person- 
of-the-quarter  listing  for  the  past  few 
years.  Who  is  on  it?  Typically  listed  is  the 
person  who  put  out  the  worst  fire.  What 
about  people  who  avoided  the  fires  in  the 
first  place?  Therein  lie  the  problems  with 
good  risk  management:  people  who  avoid fires 
do  not  get  noticed,  and  the  risks  they  avoid 
do  not  get  documented. 

Risks  that  are  well  understood  and 
controlled  tend  not  to  become  full-blown 
problems,  and  thus  are  rarely  documented 
in  risk  databases.  To  this  day,  some  people 
mistakenly  believe  the  millions  of  dollars 
spent  on  Year  2000  mitigation  were  wast¬ 
ed  because  “nothing  bad  happened.”  This 
is  the  irony:  If  people  die  or  property  is 
destroyed,  then  preventative  measures  are 
deemed  inadequate;  if  nobody  is  hurt  and 
nothing  is  destroyed,  then  preventative 
measures  are  deemed  valueless! 

We  can  do  a  lot  of  damage  in  the  name 
of  process  and  standardisation.  Some  things 
lend  themselves  well  to  both  such  as 
building  a  car  on  an  assembly  line.  Some 
things  do  not  such  as  creative,  knowledge- 
based  work  like  design  and  management. 
Yet  we  sometimes  delude  ourselves  by  cre¬ 
ating  templates  for  something  like  a  risk 
management  plan.  Look  carefully  at  such 
templates:  80  percent  of  the  outline  tends 
to  be  boilerplate  or  context  setting.  The 
meat  is  contained  in  sections  that  com¬ 


prise  only  20  percent  of  the  table  of  con¬ 
tents  entries.  What  does  the  template  tell 
you  about  those  meaty  sections?  Almost 
nothing!  The  real  meat  of  a  risk  manage¬ 
ment  plan  —  assembling  a  qualified  team, 
devising  ways  to  discover  risks,  devising 
methods  of  quantifying  or  categorizing 
the  risks,  and  monitoring  the  risks  —  can¬ 
not  be  completed  by  simply  following  a 
checklist. 

In  contrast,  template  instructions  for 
the  non-meaty  sections  tend  to  be  far 
more  explicit  (e.g.,  “state  your  funding 
authorization  by  appropriation  for  each 
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fiscal  year”).  Usually,  this  information  is 
readily  available  and  easily  culled  from 
program  management  plans,  status 
reports,  organizational  charts,  etc.  We 
delude  ourselves  into  thinking  that  a  plan 
is  80  percent  complete  when  in  fact  we  are 
just  getting  started. 

There  is  a  subtle  yet  critical  message 
implied  in  the  above:  Nobody  can  give  you 
a  simple  risk  checklist.  The  reality  is,  when 
people  want  to  learn/know  how  to  do  risk 
management ,  they  are  looking  for  Dick-and- 
Jane  instructions  for  the  meaty  20  percent. 
That  is,  they  are  specifically  looking  for 
detailed  steps  on  those  things  that  cannot 
be  determined  in  advance  by  someone 
who  is  not  intimately  familiar  with  the 
project  and  its  domain  and  environment. 


Simply  put,  they  are  looking  for  steps, 
words  like  “go  to  the  financial  department 
and  get  last  month’s  numbers  and  look  for 
expenditures  that  lag  tire  fiscal  year  spend¬ 
ing  plan.”  They  do  not  want  tasks  like 
“monitor  the  expenditures  to  verify 
claimed  accomplishments.”  The  message  I 
get  is,  “Do  not  tell  me  what  to  think  about 
or  investigate,  tell  me  exactly  who  to  see, 
exactly  what  to  ask,  exactly  what  to  record, 
and  exactly  what  to  do  about  it.  Don’t  make 
this  hard  -  just  tell  me  exactly  what  to  do.” 

There  can  be  value  added  from  a  tem¬ 
plate.  But  this  is  far  more  likely  when  the 
template  is  based  on  a  process  or  proce¬ 
dure  that  is  absolutely  relevant  to  the  pro¬ 
gram.  For  example,  if  you  are  managing 
an  avionics  modernization  project,  your 
risk  plan  template  should  come  from 
another  avionics  modernization  project. 
Not  only  that,  but  also  the  template 
should  have  been  assessed  and  revised  by 
the  last  project.  This  feedback  loop  is  crit¬ 
ical!  If  there  was  no  feedback,  then  you 
have  no  idea  if  the  template’s  prior  users 
benefited  from  it  or  not.  In  the  worst  case, 
the  very  template  you  propose  to  use  may 
have  hindered  their  ability  to  discover, 
quantify,  categorize,  prioritize,  and  man¬ 
age  project  risks  -  and  you  do  not  know 
that!  Ideally,  the  prior  users  reviewed  and 
updated  their  risk  management  plan 
throughout  their  project,  and  all  of  their 
lessons  learned  were  captured  -  you 
should  do  this,  too. 

Speaking  of  lessons  learned,  I  am 
often  asked  for  databases  of  risks,  or  more 
simply,  where  an  interested  party  should 
look  for  risks  experienced  in  past  pro¬ 
grams.  The  answer  always  disappoints  the 
inquirer  for  two  reasons.  First,  the  histori¬ 
cal  data  that  exists  is  typically  a  list  of  prob¬ 
lems,  not  risks.  Risks  are  undesirable  events 
that  could  happen:  The  concern  over  pos¬ 
sible  glitches  associated  with  Year  2000  is 
a  great  example.  Problems  are  risks  that 
came  to  fruition.  Problems  are  well  docu¬ 
mented  in  post-mortem  analyses.  But 
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good  risk  management  —  risk  that  did  not 
turn  into  problems  —  is  forgotten. 

Second,  risks  -  and  even  problems  - 
experienced  by  past  programs  are  tuned  for 
die  environment  diat  existed  for  that  pro¬ 
gram  and  die  unique  circumstances  of 
diat  program.  What  may  have  been  a  high- 
priority  risk  for  a  past  program  may  not  be 
wordi  your  investment  of  resources  to 
monitor  or  track.  Most  people  who 
request  lessons  learned  do  not  really  want 
a  database  anyway.  They  want  the  15  or  20 
items  from  die  database  that  are  most  like¬ 
ly  to  happen  to  diem.  And  they  do  not 
want  to  read  hundreds  of  items  to  find 
those  15  to  20  nuggets.  They  are  really 
asking  me  for  a  five-minute  answer  to  a 
two-week  question. 

That  is  not  to  say  that  there  is  no  value 
added  in  researching  history.  My  experi¬ 
ences  show  that  there  is  a  fertile  ground 
for  finding  risks  —  we  know  diis  because 
problems  have  consistently  arisen  from 
diese  areas.  I  have  learned  to  focus  some 
risk  identification  energy  on  diree  areas  (if 
diey  are  present  in  a  project):  test  and  inte¬ 
gration  hardware,  interfaces,  and  reused 
code. 

•  Test  and  integration  hardware  tends  to 
be  a  capacity-constraining  resource.  If 
you  have  a  system  or  software  integra¬ 
tion  lab  (SIL),  you  have  a  potential 
resource  conflict.  Many  efforts  in  the 
program  seem  to  demand  SIL  time 
simultaneously,  and  usually  the  soft¬ 
ware  developers  do  not  have  top  prior¬ 
ity.  I  worked  on  a  program  where  the 
same  test  hardware  was  used  to  vali¬ 
date  test  software  and  to  test  hardware 
diat  was  about  to  be  sold  to  the  gov¬ 
ernment.  Needless  to  say,  the  chance 
to  generate  revenue  trumped  the  soft¬ 
ware  developer’s  needs  until  we  were 
able  to  prove  that  the  impending 
delays  to  the  project  would  negatively 
affect  the  contractor’s  bottom  line  by 
more  dian  a  little  delay  in  cash  flow. 
While  working  on  a  different  program, 
I  discovered  that  the  developer’s 
detailed  schedules  required  over  30 
hours  per  day  in  the  SIL  to  meet  the 
schedule.  Scheduling  tools  are  great, 
but  they  fail  when  you  disable  or 
ignore  the  resource  conflict  warnings. 

•  Interfaces  are  historically  a  source  of 
error,  and  therefore  risk.  A  recent  big 
example  was  the  Mars  Climate  Orbiter 
diat  crashed  into  the  planet  in  1999 
because  one  group  coded  as  if  the 
measurement  were  in  feet  while  the 
other  coded  as  if  it  were  in  meters. 
Most  bugs  in  a  program  are  problems 
found  while  integrating  modules  or 
communicating  between  objects.  On  a 


grander  scale  in  systems  of  systems, 
die  biggest  risks  are  where  die  inde¬ 
pendently  built  systems  must  interface. 
System  test  engineers  always  praise  a 
good  interface  control  document 
(ICD)  more  than  the  project  managers 
bemoan  the  ICD’s  cost.  We  have  a 
proverb  that  “good  fences  make  good 
neighbors”  and  the  same  is  true  in 
software:  If  everyone  knows  the 
boundary  conditions  and  interfaces, 
tilings  go  much  smoodier.  The  hard 
part  is  resisting  die  temptation  to  cut 
or  minimize  the  typically  large  expense 
of  creating  good  ICDs.  ICDs  are  used 
for  inter- system  interfaces,  but  there 
are  analogous  -  and  equally  valuable  - 
design  products  that  should  describe 
die  intra- system  interfaces  in  detail. 

•  Reused  code,  which  includes  commer¬ 
cial  off-the-shelf  code,  is  often  sold  to 
die  program  as  a  means  of  drastically 
reducing  development  and  test  costs. 
Code  reuse  can  certainly  reduce  costs, 
but  only  within  the  very  narrow  cir¬ 
cumstances  where  you  make  absolute¬ 
ly  no  changes  to  the  code,  and  you  use 
it  for  exactly  and  only  die  purpose  for 
which  it  was  designed.  Many  potential¬ 
ly  dangerous  commercial  products  like 
pesticides  now  carry  a  standard  warn¬ 
ing  such  as  “Use  of  this  product  in  a 
manner  other  than  described  below  is 
a  violation  of  federal  law.”  Yes,  the 
spray  is  flammable  -  no,  you  should 
not  use  it  to  light  your  barbeque  grill. 
A  similar  warning  should  accompany 
all  attempts  to  reuse  code,  albeit  only  a 
warning  diat  it  violates  sound  reuse 
strategy,  and  maybe  die  laws  of  good 
sense.  It  is  not  a  bad  idea  to  reuse 
code,  but  you  have  to  accept  the  limi¬ 
tations  when  you  do.  If  your  plans  call 
for  reusing  code  and  you  are  assuming 
substantial  time  and  cost  savings  or 
test  simplification,  you  had  better  not 
tinker  with  the  reused  code  (or  code 
products)  in  any  way,  or  you  violate 
your  plan/assumption  and  incur  risk. 
Of  course,  the  risk  manager  must  look 
beyond  these  three  areas,  and  must  apply 
knowledge  of  the  project’s  details  to 
determine  whether  any  of  those  three 
areas  are  applicable  and  worthy  of  invest¬ 
ing  resources. 

Risk  management  is  much  like  being 
die  manager  of  a  mutual  fund  or  a  stock 
analyst  on  Wall  Street.  Risk  managers  are 
asked  to  peer  into  the  future  —  to  make 
predictions  with  better-dian-average  accu¬ 
racy  —  to  not  only  be  right,  but  to  know 
what  to  do  when  they  are  right.  Risk  man¬ 
agement  goes  beyond  predicting  risk;  it 
also  demands  planning  to  handle  the  risk 


once  it  materializes.  (As  a  side  note,  drink 
of  how  well  paid  mutual  fund  managers 
and  Wall  Street  stock  analysts  are,  espe¬ 
cially  the  successful  ones!) 

How  do  fund  managers  and  analysts 
become  successful?  They  dig  into  the 
details  of  a  company.  They  may  not  have 
complete  data  because  die  company  may 
not  release  any  more  than  the  minimum 
required  by  law.  Yet  the  manager  can 
assemble  current  information  about  this 
particular  company,  as  well  as  information 
from  its  recent  and  not-so-recent  past. 
Information  can  be  gathered  about  similar 
companies  over  time,  and  about  the  seg¬ 
ment  of  the  economy  that  affects  this 
company.  This  information  can  then  be 
used  to  make  an  educated  guess  at  future 
earnings,  profits,  and  trends.  In  other 
words,  they  develop  detailed  knowledge 
about  the  specific  company,  and  compare 
it  with  a  solid  general  knowledge  about  the 
industry  and  the  economy.  This  helps 
them  more  accurately  foresee  profitability, 
which  can  then  be  used  to  make  sound 
investment  decisions. 

This  is  the  essence  of  risk  manage¬ 
ment!  The  risk  manager  combines  detailed 
knowledge  of  the  project  widi  general 
knowledge  of  die  technical  domain  and 
the  acquisition  environment  to  foresee 
potential  undesirable  events,  and  to  plan 
and  take  actions  accordingly. 

Asking  a  complete  novice  to  do  risk 
management  is,  well,  risky.  Risk  manage¬ 
ment  involves  thoughtful,  determined, 
and  creative  work  to  implement  the  fol¬ 
lowing  eight-step  process. 

Step  I :  Get  Time  to  Do  Risk 
Management 

If  you  are  spending  95  percent  of  your 
time  doing  day-to-day  operations,  you  do 
not  have  enough  time  to  sit  and  drink  (or 
plan  or  just  be  creative).  You  need  slack 
time  —  that  is,  time  away  from  operations 
-  to  plan  and  think.  For  a  great  discussion 
on  why,  read  Tom  DeMarco’s  book  Slack 
[1],  It  even  contains  a  few  chapters  on  risk 
management.  Sometimes,  this  seemingly 
simple  step  can  be  the  hardest  part.  Next 
comes  the  creative  part. 

Step  2:  Plan  Your  Risk 
Management  Program 

What  method  will  you  use  to 
discover/elicit  risks?  Who  will  help?  (Hint: 
you  need  those  people  who  are  intimately 
familiar  widi  die  project,  die  domain,  and 
the  environment.)  What  are  the  desired 
outputs  of  your  risk  analysis?  How  will 
you  categorize  or  quantify  risk?  What 
information  must  be  recorded  for  each 
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risk?  Who  will  use  the  data  and  how?  Now 
comes  more  creativity  (problem  solving) 
and  some  tedium. 

Step  3:  Identify  Risks 

Gather  the  team  and  identify  potential 
risks.  Remember  that  die  team  should 
consist  of  people  with  lots  of  project  and 
domain  experience.  These  people  tend  to 
be  senior  members  and  are  very  busy,  so 
these  identification  sessions  should  be 
short  and  controlled.  Excellent  adminis¬ 
trative  support  is  absolutely  necessary!  So 
is  follow-up  and  coordination  of  results. 
For  each  risk  identified,  the  team  should 
describe  what  data  they  need  to  assess  the 
risk.  Much  of  that  data  will  probably  not 
be  available  at  this  meeting,  which  is  okay. 
This  first  session  is  identification  only. 

Step  4:  Assess  Risks 

The  risk  team  does  risk  assessment.  It 
involves  a  facilitator  doing  lots  of 
research  and  legwork  before  another 
meeting  with  the  experts.  Once  the  data  is 
available  and  pre -distributed,  the  team  can 
reconvene  to  assess  probabilities  and 
impacts,  determine  indicators  that  a  risk 
may  be  coming  true ,  and  prioritize  the  risks 
according  to  the  documented  procedure. 
The  indicators  are  used  to  select  metrics 
so  the  decision-maker  can  be  proactive 
when  choosing  whether  to  implement 
handling  strategies. 

Step  5:  Plan  to  Handle  Risks 

With  the  decision-maker  and  the  team, 
decide  how  each  risk  will  be  handled. 
Determine  what,  if  any,  mitigation  efforts 
are  prudent;  what  alternative  approaches 
or  procedures  are  available;  and/ or  how  to 
share  the  risk.  It  is  a  good  idea  to  identify 
thresholds  (or  trigger  points)  associated 
with  die  metrics  selected  in  Step  4  so  it  is 
easier  to  initiate  action. 

Step  6:  Monitor  Risks 

Conduct  operations  and  periodically 
check  to  see  if  any  of  the  risks  show  signs 
of  turning  into  problems,  or  if  any  of  the 
risks  change  because  of  die  dynamics  of 
project  and  environment.  This  period 
could  be  daily  or  weekly  or  something  dif¬ 
ferent,  depending  on  how  dynamic  the 
project  and  environment  are. 

Step  7:Account  for  Changes 
in  the  Environment  and 
Project 

Periodically  go  back  to  Step  3.  This  period 
could  be  weekly  or  monthly  or  somediing 
different,  depending  on  how  dynamic  the 
project  and  environment  are. 


Step  8:  Improve  Your  Risk 
Management  Process 

Periodically  go  back  to  Step  2.  This  period 
could  be  quarterly  or  annually  or  some¬ 
diing  different,  depending  on  how  suc¬ 
cessful  your  program  is  at  giving  sufficient 
notice  of  things  drat  may  go  wrong.  This 
is  the  part  diat  everyone  hates,  but  it  is  the 
critical  feedback  loop  diat  improves  the 
process  —  for  you  and  for  the  next  project 
diat  uses  your  project  as  a  template. 

General  Ideas 

Here  are  some  general  ideas  on  risks.  They 
must  be  general  because  I  do  not  (and 
cannot)  know  the  details  of  every  reader’s 
situation. 

•  If  you  cannot  assign  a  probability, 
assess  an  impact,  or  draft  a  unique 
action  plan,  dien  the  risk  you  have 
identified  is  too  generic,  or  not  a  risk  at 
all.  For  example,  stating  that  the  risk  is 
“our  budget  will  get  cut”  is  meaning¬ 
less  because  you  cannot  say  what  the 
impact  is  or  what  you  would  do  about 
it.  A  better  risk  would  be  “next  year’s 
budget  will  be  cut  by  5  percent,  which 
means  we  cannot  fully  fund  long-lead 
spares.”  Document  why  you  chose  the 
numbers  you  did.  Why  5  percent  and 
not  8  percent  or  2  percent?  Why 
impact  spares  and  not  tech  orders? 

•  If  a  risk  is  a  near-certainty,  dien  it  is 
not  a  risk,  it  is  something  that  the  pro¬ 
ject’s  execution  plan  should  already 
address.  Does  it? 

•  Risks  should  be  prioritized  according  to 
an  agreed-upon  scheme.  The  risk  team 
may  track  100  risks.  Project  managers 
may  only  have  time  to  track  the  top  10. 
Of  those,  the  senior  acquisition  offi¬ 
cials  probably  have  time  and  attention 
for  only  die  top  two  or  three.  Know 
how  these  lists  will  be  derived.  Are  they 
based  on  probability  of  occurrence? 
Are  they  based  on  severity  of  impact  if 
they  do  occur?  Are  they  based  on  some 
combination  of  die  two? 

•  A  top  10  list  should  have  exactly  10 
items.  Having  15  different  No.  1  priori¬ 
ty  items  may  look  good  when  spread¬ 
ing  the  wealth  for  performance  review 
bullets,  but  it  does  nothing  for  helping 
senior  people  prioritize  their  time  and 
die  favors  they  would  like  to  call  in. 

•  Good  risk  descriptions  include  indica¬ 
tors,  or  some  method  of  foreseeing 
diat  the  risk  may  actually  be  coming 
true.  The  better  these  indicators  are, 
die  better  you  can  prepare  the  contin¬ 
gency  plans. 

Finally,  there  are  many  approaches  and 
processes  to  manage  risk.  An  Internet 


search  will  turn  up  dozens.  But  remember 
the  rule  of  domain  applicability:  If  the 
risk  management  process  was  built  by 
those  making  and  assembling  automo¬ 
biles,  it  may  not  be  well  suited  for  a  differ¬ 
ent  environment  such  as  software  devel¬ 
opment.  Risk  management,  when  done 
correctly,  consumes  die  time  of  the  most 
experienced,  most  project-knowledgeable 
people  who  also  happen  to  be  die  busiest 
and  highest-paid.  However,  die  cost  and 
effort  to  prevent  a  fire  is  almost  always  far 
less  than  the  cost  and  effort  to  rebuild 
after  the  fire  is  out.^ 
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Note 

1 .  The  views  expressed  in  diis  article  are 
those  of  die  author  and  do  not  neces¬ 
sarily  reflect  the  official  policy  or  posi¬ 
tion  of  the  Air  Force,  Department  of 
Defense,  or  the  U.S.  government 
agency. 
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